Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flip MeteoBase data #185

Merged
merged 5 commits into from
Jun 6, 2023
Merged

Flip MeteoBase data #185

merged 5 commits into from
Jun 6, 2023

Conversation

bdestombe
Copy link
Collaborator

No description provided.

@bdestombe bdestombe requested a review from martinvonk June 5, 2023 10:41
@martinvonk
Copy link
Collaborator

martinvonk commented Jun 5, 2023

Hi @bdestombe,

Thanks for the PR. Can you tell me which bug this solves? And why the data has to be flipped?

Also, on the nodata default argument; the DataArray's attributes contains the NODATA_value and if this is replaced by np.nan this value is not valid anymore. So I like the default argument to be False such that filling this np.nan is a conscious decision of the user. Or we have to replace the NODATA_value in the metadata.

@bdestombe
Copy link
Collaborator Author

Hi Martin,
The image was flipped vertically, as was clearly visible in the evapotranpiration product.

Nans were invented exactly for this purpose, so why not use them? The nodata value is used when working with integers. And some file formats require working with integers such that nodata values are inherently required. That is not the case here, as all the products contain floats and xarray manages nans quite well.

When exported to tiffs the nans get correctly propagated otherwise you'd manually have to assign the no data value via rioxarray. Furthermore, the xarray plotting utilities work well with nans.

But you are right, the nodata_value now has no further meaning. I'll remove it in the PR.

Thanks for your feedback!

@martinvonk
Copy link
Collaborator

martinvonk commented Jun 5, 2023

The image was flipped vertically, as was clearly visible in the evapotranpiration product.

Is this also the case with the other products? Or is this a problem with MeteoBase?

@martinvonk
Copy link
Collaborator

Nans were invented exactly for this purpose, so why not use them?

I'm fine with using NaNs, as long as the metadata stays consistent :) So the current solution is fine with me.

@bdestombe
Copy link
Collaborator Author

bdestombe commented Jun 5, 2023

The image was flipped vertically, as was clearly visible in the evapotranpiration product.

Is this also the case with the other products? Or is this a problem with MeteoBase?

Good question.. I am not sure how to check this. Any suggestions?

@bdestombe
Copy link
Collaborator Author

Both evapotranspiration and verdampingstekort are now orientated correctly. I'm not sure how to check precepitation. And Manning en Penman only contain nodata values on the dataset that I have at hand.

@martinvonk
Copy link
Collaborator

martinvonk commented Jun 5, 2023

We could maybe compare the precipitation dataset to the KNMI radar dataset (#166). See how that one compares to the flipped version.

The dataset for meteobase does not contain NODATA for the Makkink and Penman evaporation: https://github.com/ArtesiaWater/nlmod/blob/main/tests/data/Meteobase_ASCII_test.zip

@bdestombe
Copy link
Collaborator Author

Ah indeed! It shows data, but its too small with too little resolution to distinguish features.

The produced geotiffs of your test data are attached.
out_test.zip

@martinvonk
Copy link
Collaborator

I think it is fine now so we can merge this.
I now flip the data when constructing the y-coordinates from the metadata, instead of flipping the data when reading the ASCII.

I wasn't able to confirm the precipitation dataset with the KNMI data. KNMI uses another stereographic projection and I don't have time for that at the moment.

@martinvonk martinvonk changed the title Flip meteobase data and bug in loading data Flip MeteoBase data Jun 6, 2023
@martinvonk martinvonk merged commit e218e01 into dev Jun 6, 2023
@dbrakenhoff dbrakenhoff deleted the meteobase_flipped branch August 24, 2023 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants